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Abstract — An analog source is to be transmitted across a 
Gaussian channel in more than one channel use per source 
symbol. This paper derives a lower bound on the asymptotic 
mean squared error for a strategy that consists of repeatedly 
quantizing the source, transmitting the quantizer outputs in the 
first channel uses, and sending the remaining quantization error 
uncoded in the last channel use. The bound coincides with the 
performance achieved by a suboptimal decoder studied by the 
authors in a previous paper, thereby establishing that the bound 
is tight. 

I. Introduction 

This paper gives performance limits of a certain class of 
encoders for the transmission of a discrete-time memoryless 
analog source across a discrete-time memoryless Gaussian 
channel, where the channel can be used n times for each 
source symbol. The parameter n is arbitrary but fixed, given 
as part of the problem statement. 

It is well known that if the channel noise has variance az 2 
then the average transmit power P and the average mean- 
squared error D of any communication scheme for this sce- 
nario are related by 



R{D) < nC(P), 



(1) 



where R(D) is the rate-distortion function of the source under 
squared-error distortion and C(P) — 0.51og(l + P/az 2 ) is 
the cost-constrained capacity of the channel (see e.g. fT\). If 
the source has finite differential entropy h(S) then the rate- 
distortion function satisfies 

R(D) > h(S) - 0.51og(27reL>). 

Applying this bound to ((TJ and inserting the capacity formula 
yields 

2 2/l ( s ) 

D>— (l + P/a z 2 )- n (2) 

27re 

or D > c(l + SNR)~", where we have defined SNR = P/az 2 
and c = 2 2h( - s ^ /2we. As SNR — !• oo, the squared error 
distortion scales thus at best as SNR _n . 

In this paper we study a communication scheme for this sce- 
nario that is extremely simple to implement and has minimal 
delay, in the sense that it encodes and transmits a single source 
symbol at a time. It works by quantizing the source and then 
repeatedly quantizing the quantization error; the quantized 



points are sent across the first n — 1 channel uses and the 
last quantization error is sent uncoded in the n th channel use. 

We show that no matter how the quantization resolution 
is chosen (as a function of the SNR) and regardless of the 
decoder used, the mean squared error achieved by this scheme 
cannot decay faster than SNR~"(log SNR)™ -1 . This asymptotic 
lower bound coincides with the achievable performance of 
a suboptimal decoder that we have studied in a previous 
paper O; it is therefore tight. 

Transmission schemes of the kind proposed here have been 
considered before. Indeed, one of the first schemes to transmit 
an analog source across two uses of a Gaussian channel 
was suggested by Shannon 0. Generalizing Shannon's ideas, 
Wozencraft and Jacobs [4] provided the foundations to analyze 
source-channel mappings as curves in n-dimensional space. 
Ziv |5] found important theoretical limitations of such map- 
pings. 

Much of the later work is due to Ramstad and his coauthors 
(see e.g. J6], Q, O). A proof that the performance of 
minimal-delay codes is strictly smaller than that of codes with 
unrestricted delay when n > 1 was given in 2008 by Ingber 
et al. @. 

For n = 2, the presented scheme is almost identical to the 
HSQLC scheme by Coward [ 10], which uses a numerically 
optimized quantizer, transmitter and receiver to minimize the 
mean-squared error (MSE) for finite values of the SNR. 
Coward conjectured that the right strategy for n > 2 would be 
to repeatedly quantize the quantization error from the previous 
step, which is exactly what we do here. 

Another closely related communication scheme is the shift- 
map scheme due to Chen and Wornell ifTTI . Vaishampayan 
and Costa lfl2l showed in their analysis that it achieves a 
squared error that scales as SNR _n+e for any fixed e > if 
the relevant parameters are chosen correctly as a function of 
the SNR. Up to rotation and a different constellation shaping, 
the shift-map scheme is in fact virtually identical to the one 
used here, a fact that was pointed out recently by Taherzadeh 
and Khandani [ 1 3 1 . This suggests that its performance is also 
limited by the bound derived here. 

Other hybrid schemes, such as the one by Shamai et al. lfl4l 
or that of Mittal and Phamdo [ 15 1, use long block codes for the 
digital phase and are therefore not directly comparable with 



minimum delay schemes. 

The rest of this paper is organized as follows. Section [TT] 
describes our transmission strategy, in Section [Til] we quote 
the achievability result from our previous paper, and finally 
Section [TV] contains our derivation of the mean squared error 
lower bound. 

II. Proposed Communication Scheme 

To encode a single source letter S into n channel input 
symbols X\, . . . , X n we proceed as follows. Define Eq = S 
and recursively compute the pairs (Qi, Ei) as 

Qi = -smt{pEi-x) 

Ei = p(Ei^ - Qi) (3) 
re—l where int(x) is the unique integer i such 



for i = l, 
that 



x € 



1 



1 



^2'* + 2 



and f3 is a scaling factor that grows with the SNR in a way 
to be determined later. Qi is thus a quantized version of 
the previous round's quantization error, and Ei is the new 
quantization error scaled up to lie in [—1/2, 1/2). Note that 
the map S H> (Q\, Q n -U En-i) is one-to-one with the 
inverse given by 



n— 1 

5 = E^T^ 



1=1 



-E n 



(4) 



We determine the channel input symbols from the Qi 
and from E n _x according to 



Xi 



X n — 



p 



-Qi for i = 1, . . . , n — 1 and 



l_P_ 

<JE' 



;E n - 



(5) 



where erg 2 = Var(-E n _i), and where 5 > is some small 
number. As shown in Q, this ensures that E[X?] < P for 
all i and for j3 > j3o (where (3q depends on 5), Since we are 
interested in the large SNR regime and since we have defined 
(3 to grow with the SNR, we can assume for the remainder 
that the power constraint is satisfied. 

III. Achievable Performance 

In this section we quote the relevant results of our earlier 
paper [2|, which imply that a suboptimal decoder achieves a 
mean squared error that scales at least as SNR~"(log SNR)™ -1 . 
While we only considered Gaussian sources in that paper, we 
actually never used the distribution of the source when we 
derived the bounds there, so they hold for general continuous 
sources of bounded variance. 



A. Suboptimal Decoder 

The encoder outputs Xi are transmitted across the channel, 
producing at the channel output the symbols 



Yj — Xi 



1, 



where the Zi are iid Gaussian random variables of vari- 
ance <jz 2 ■ To estimate S from Yi, . . ., Y n , the decoder first 
computes separate estimates Q\, Q n -i and E n —\, and 
then combines them to obtain the final estimate S. 

To estimate the Qi we use a maximum likelihood (ML) 
decoder, which yields the minimum distance estimate 



Q l 



l 



• arg mm 



P 



5/3 



Yi 



(6) 



To estimate B n _i, we use a linear minimum mean-square error 
(LMMSE) estimator (see e.g. Scharf [16] Section 8.3]), which 
computes 

E[E n -iY r 



Finally we use to obtain 

n—l 



Era 



1 



n — l 



En 



(7) 



(8) 



B. Upper Bounds on the Mean Squared Error 

Using the suboptimal decoder described in the previous 
section, E[(S — S) 2 } can be broken up into contributions due 
to the errors in decoding Qi and E n -i as follows. From 
and ([H]), the difference between S and S is 



S-S 



n—l j 
i=l r 



j{Qi - Qi) + -——[{En-l - E n -i). 



The error terms Qi — Qi depend only on the noise of the 
respective channel uses and are therefore independent of each 
other and of _E„_i —E n -i, so we can write the error variance 
componentwise as 



1 



where S Qti d = E[(Q 4 - C?,) 2 ] and £ E = E[(^ n _ x - E n -i) 2 }. 

The following two Lemmata, taken directly from [2|, give 
upper bounds on the two types of errors. (The O-notation is 
defined in Appendix lAl) 

Lemma 1: For each i = 1, . . . , n — 1, 



~'Q,i 



dcf - 



/32(n- 



(9) 



£q,i € O (exp{-fcsNR//3 2 }) , 



(10) 



where SNR = Pjaz 2 and k > does not depend on SNR. 
Lemma 2: The estimation error of B n _i satisfies 



£ E /^ {n ~ X) € 0(SNR" 1 j3- 2{n - 1) ). 



(11) 



From Lemma [T] (3 2 should scale less than linearly in SNR, 
otherwise the upper bound would be constant. We therefore let 



(3 2 = SNR 1 e , where e is an arbitrary, strictly positive function 
of SNR. From (ITOb and ( fTTT i we have then 



and 



£<g,i £ 0(exp{-fcSNR £ }) 



Setting e(SNR) = log((n/fc) log SNR)/ log SNR we find that 
the performance achieved by the suboptimal decoder satisfies 

E[(S - S) 2 } G ©(SNR-'^logSNR)"- 1 ). 

The next section shows that this is the best achievable scaling 
for the given encoder, even if an optimal decoder is used. 

IV. Distortion Lower Bounds 

The goal of this section is to lower bound the scaling of the 
mean squared error of the transmission strategy described in 
Section HH 

Throughout this section we assume (3 2 = SNR 1_e , where 
e = e(SNR) is a positive function of SNR. This results in no 
loss of generality, since for an arbitrary positive function / we 

can set c(snr) = l-log(/(sNR))/logSNR to get /3 2 (snr) = 

/(SNR). 

Note that by (01, the Qi are completely determined by S. 
In this section, with a slight abuse of notation, we therefore 
write Qi(s) to denote the value of Qi when S = s. We use 
Ei(s) and Xi(s) in a similar manner. Furthermore, we define 
X( S ) = (X 1 (,s),...,X„( S )). 

The following result, adapted from Ziv |5|, is a key ingre- 
dient in the proofs of the lemmas that follow. 

Lemma 3: Consider a communication system where a con- 
tinuous-valued source S is encoded into an n-dimensional 
vector X(5), sent across n independent parallel AWGN chan- 
nels with noise variance az 2 , and decoded at the receiver to 
produce an estimate S. If the density ps of the source is such 
that there exists an interval [A, B] and a number p m i n > 
such that ps(s) > p m in whenever s <E [A, B], then for any 
A G [0, B — A) the mean squared error incurred by the 
communication system satisfies 

/A\ 2 f B ~ A 

n(S~S) 2 }> Pmin (-j J Q(d(s,A)/2a z )ds, (12) 
where d(s, A) d = ||X(s) - X(s + A)|| and 



Q(x) 



/ (l/^)exp{-C 2 /2}^. 

J X 



Proof: See Appendix IE1 ■ 
The next two lemmata provide two different asymptotic 

lower bounds on the mean squared error of our transmission 

strategy, each of which is tighter for a different class of e. 

They hold regardless of the decoder used. (The fi-notation is 

defined in Appendix lAl) 

Lemma 4: For an arbitrary function e(SNR) > 0, the mean 

squared error satisfies 

E[(5 - S) 2 } G ft(SNR~™ +( ™~ 1)£ ). 



Lemma 5: For an arbitrary function e(SNR) > 0, the mean 
squared error satisfies 



E[(5-5) 2 ] G n(SNR 



-l+c/2 



exp{-SNR e /fc}) 



where k > does not depend on SNR. 

Discussion: An immediate consequence of the lemmata is 
that the theoretically optimal scaling SNR - " is not achievable 
with the given encoding strategy: by Lemma |4] this would 
require e = 0, but following Lemma [5] the scaling is at best 
SNR -1 if e = 0. More generally, which one of the two lower 
bounds decays more slowly and is therefore tighter depends 
on the scaling of e(SNR). How to choose e(SNR) optimally 
will be the subject of Theorem [7] 

Proof of Lemma® Assume A G [0, and define 
for j G Z 



J A = 



1 



1 



It can be verified from (f3]) that if s G X A for some j, the 
following properties hold: 1) Qi(s) = Qi(s + A) for i = 1, 
n-1, and2)£„_i(s + A)-i;„_i(s) = ^"~ 1 A . From Q 
it follows that s G Tf implies d(s, A) = ^ P/a E 2 /3 n ^ 1 A. 
We now apply Lemma [3] and restrict the integral to the 



setV(A) d = [A,B- 
relaxed to give 



A) D Ujez -^j- l° wer bound is then 



E[{S - S) 2 } > ^A 2 Q( v /sNR/a B 2 /3"- 1 A/2) / ds 



Letting A = l/(\/SNR/3" _1 ) and j3 2 = SNR 1 



yields 



E[(^-5) 2 ] > ^snr- 



i+(n— l)e/ 



{2a E ) 



ds. 



V(A) 



The proof is almost complete, but we still have to show that 
/^(A) ^ s can t> e bounded below by a constant for large SNR. 
The length of a single interval Xf is - A. Within 

[A, B - A) there are (B - A - A)/?™" 1 such intervals. The 
total length of all intervals X A in [A, B — A) is therefore 

/ ds = (B- A- A)(1-^"^ 1 A), 

which, for the given values of f3 and A, converges to B — A for 
SNR — > 00 and thus can be lower bounded by a constant for 
SNR greater than some SNR . With this, the proof is complete. 

■ 

Proof of Lemma\5\ Observe first that (f3]l implies Qi(s + 
P^ 1 ) = Qi(s) + ft- 1 and Ei(s + = S x (s). Since all 
Qi and Ei for i > 2 are by recursion a function of E\ only, 

Q^s) = Q t ( s + p- 1 ) for i = 2 n - 1, and J5 n _i(s) = 

S n _i(s + Consequently, Xi(s) = Xi(s + for all 

i = 2, . . . , n. By (01 and the above, the Euclidean distance 
between X(s) and X(s + /3 _1 ) is therefore 



P 



rlQiM-Qi^ + ZT 1 )! 



p 



(13) 



We now apply Lemma [3] with A = /3 _1 . The parameter /3 
will be chosen to increase with SNR, therefore Ae [0, B — A) 
holds for sufficiently large SNR. 

Using ( fT3] l, the resulting bound on the mean squared error 

is 

SNR /3 -1 " 



5 2 



Replacing (3 2 — SNR 1 e and using the fact that Q(x) con- 

» (cf. IfTTl ) we obtain 



verges to exp{— x 2 /2}/v27ra for x - 

E[(S ~ S) 2 } > cSNR- 1+£ / 2 exp{-SNR7fc} 

for sufficiently large SNR, with c and k positive constants that 
do not depend on SNR, thus proving the lemma. ■ 

The following lemma will be used to prove Theorem [7] the 
main result of this paper. 

Lemma 6: Define W(x) to be the function that satisfies 
W(x)e w ( x ' = x for x > 0. This function is well defined 
and is sometimes called the Lambert W -function |[T8l . Then 
for SNR > 1 and arbitrary real constants a, b > 0, and k > 0, 



if and only if 



SNR' 



SNR 



a-\-be 



exp{-SNR e //c}, 



bkW(SNR- a/b /bk). 



(14) 



(15) 



Proof: Let SNR > 1, Since SNR a+bc is strictly increasing 
and exp{— SNR e /fc} is strictly decreasing in e, there is at most 
one solution to (fT4l >. Assume now SNR e is as in ( fT3T l. Then 

exp{-SNR e /fe} = exp{-bW(SNR~ a/b /bk)}. 

On the other hand, 

SNR a+be = SNR° (bkW(SNR- a/b /bk)y 

= (vF(SNR- a / b /6fc)/(SNR- a / b /fefc)) 6 . 

By definition, W{x)/x = e - w(x \ so the above is equal to 



SNR 



a-\-be 



exp{-bW(sNR- a/b /bk)}, 



which proves the claim. ■ 
The following is the main result of the paper. 
Theorem 7: For any parameter f3 and for any decoder, the 

mean squared error of the transmission strategy described in 

Section [TT] satisfies 



E[(S - S) 2 } G fi(SNR-"(log SNR 



,n-l\ 



Discussion: The asymptotic lower bound on the mean 
squared error given by the theorem coincides with the asymp- 
totic performance achieved by the suboptimal decoder in 
Section [Till the bound is therefore asymptotically tight. 

Proof of Theorem^ Define Zi(SNR,e) = SNR- n+( -"- 1 ^ 
and Z 2 (SNR,e) = SNR~ 1+£ / 2 exp{-SNR e //c}. By Lemmata 0] 
and|5l 

E[(S'-S') 2 ] G fi(max(Z 1 (SNR,e),? 2 (SNR,e))). 



The optimal parameter e(SNR) is therefore such that for 
any SNR 

max(Zi(SNR,e),Z 2 (SNR, e)) (16) 

is minimized. Now for any fixed SNR, Zi(snr, e) is increasing 
in e, and ^(snr, e) is increasing in e for < e < £ = 
log(fc/2) / log SNR and decreasing in e for e > £. The maxi- 
mum in (fTol l is therefore minimized either for e = or for e > 
£ such that h(e) = Z 2 (e)- As we have remarked earlier, e = 
leads to a worse performance than that achieved in Section Hill 
and so this cannot be the optimal parameter. We therefore have 
to choose e(SNR) such that Zi(SNR, e) = / 2 (SNR,e). Inserting 
the definitions of l\ and Z 2 and rearranging the terms yields 



SNR 



-(n-l)+(n— 3/2)e 



exp{-SNR e /A;}, 



which is of the form (TBI) with a = — (n — 1) and b = n — 3/2. 
By Lemma |U for SNR > 1, 

snr £ = (n - 3/2)fciy(sNR 2 ^ 1 /((n - 3/2)fc)). 

We now use the fact that W(x) / log x converges to 1 for x —> 
oo; this can be shown using L'Hopital's rule and because the 
derivative of W(x) is W(x)/[x(l + W(x))] (cf. Ifl8l ). 

For sufficiently large SNR, therefore, there exists a constant 
c > such that 



SNR E > c(n-3/2)fc 



2(n- 1) 



2n - 3 



logSNR-log((n-3/2)A:) 



and so SNR £ G O(logSNR). Plugging this into the bound of 
Lemma |4] we finally obtainj 

E[(S - S) 2 } G r2(sNR-"(log SNR)™" 1 ), 

and no choice of e(SNR) can improve this bound. ■ 

V. Conclusions 

We have analyzed the source-channel coding strategy of 
repeatedly quantizing an analog source and transmitting the 
quantizer outputs and the remaining quantization error un- 
coded across n Gaussian channels. We have shown that if 
the quantization resolution of the encoder is chosen optimally 
and if the optimal decoder is used then the mean squared 
error scales at best as SNR~™(log SNR) 11-1 . Furthermore, as 
our previous paper showed, a simple suboptimal decoder is 
sufficient to achieve this scaling, so the bound is tight. 

The question whether any minimal delay scheme can 
asymptotically perform better than SNR~"(log SNR)" -1 is still 
open at this time. 

Appendix A 
Asymptotic Notation 

The "O" and "f2" asymptotic notation used at various points 
in the paper is defined as follows. Let f(x) and g(x) be two 
functions defined on M. We write 

f(x) G 0(g(x)) 

'if a(x) e Sl(f(x)) and b(x) G fi(fl(*)), then a{x)b(x) m £ 
n{f[x)g(x)^). 



if and only if there exists an Xq and a constant c such that 

f(x) < cg{x) 

for all x > xq. 

Similarly, we write f(x) £ tt(x) if < is replaced by > in 
the above definition. 

Appendix B 
Proof of Ziv's Lower Bound (Lemma[3]) 

If we condition the mean squared error on S and use the 
assumption on ps we obtain 

K[(S - S) 2 } > Pmin / E[(S - S) 2 \s]ds. 

J A 

For A 6 [0, B — A] we can further bound this in two ways: 

i-B—A 



Pr[error|s + A] < Pr[|5 - S- A| > A/2 | s + A]. Applying 
this to (fT8T l and ( fT9] l and inserting the result in ( fTTT i yields 



[(5 - > p„ 



E[(S-S) 2 |s]ds 



EKS - SY\ > Pmin / E[(S-S)>]da 
/•B-A 

= p min / E[(5-5- A) 2 | s + A]ds. 



Averaging the two lower bounds yields 



E[(S-S) 2 }>^ [E[(S - S) 2 \s} + 



E[(S - S - A) 2 \s + A]jds, (17) 

and applying Markov's inequality to the expectation terms 
leads to 



E[(.S'- S')-|.s] > (^) Pr[|.S'-,S' > A/2 .s] (18) 



and 



E[{S-S- A) 2 |s + A] 

A x2 



> (-) Pr[|5-5-A|>A/2|s + A]. (19) 



Now suppose that we use the communication system in 
question for binary signaling. We want to send either s 
or s + A; at the decoder we use the estimate S to de- 
cide for s or s + A depending on which one S is closer 
to. When s is sent, the decoder makes an error only if 
\S — s\ > A/2; when s + A is sent, it makes an error only 
if \S — s — A| > A/2. The conditional error probabilities 
therefore satisfy Pr[error|s] < Pr[|5 - S\ > A/2 | s] and 



i[(s-sy j ] > Pmin [ ^ 



B—A 



P e (s,A)ds, (20) 



where P e (s,A) = (Pr[error|s] + Pr [error |s + A]) /2 is the 
average error probability. 

If s and s + A are picked with equal probability and 
transmitted across n parallel Gaussian channels as X(s) and 
X(s + A), and if d(s, A) = ||X(s)-X(s + A)||, then the error 
probability of the MAP decoder is Q(d(s, A) /2a z), a standard 
result of communication theory (see e.g. [4, Section 4.5]). 
Because the MAP decoder minimizes the error probability, 
Q(d(s, A)/2a z ) < P e (s, A), which, when inserted into d20i >. 
completes the proof. ■ 
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